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13.  ABSTRACT  (Mni$mum  700  wonts) 

Uiiique  hardware  and  novel  software  were  developed  to  study  natural  patterns  of  head  and  eye  movements 
during  inspection  and  nuinipulation  of  objects  in  nearby  3-D  space.  Emphasis  was  placed  on  natural  tasks 
in  which  subjects  looked  at  and  handled  real  objects  in  a  natural  visu^  environment  with  the  head  and 
torso  ftee  to  move  naturally.  Two  quite  different  mechanistic  models  of  gaze  control  were  tested;  (a) 
an  "on-line"  feedback  model  and  (b)  a  "single  packet"  model  that  bases  accurate  gaze  control  on  pre- 
plaiuied  patterns  of  coordinated  movements  of  the  head  and  eyes.  We  found  that  the  VOR  was  not 
tiuned-off  during  gaze-shifts,  as  currently  believed,  but  that  it  was  not  effective  in  compensating  for 
abrupt  displacements  of  the  head  as  had  been  believed  previously.  The  speed  and  accuracy  of  bunon 
presses  produced  by  tapping  and  the  correlation  of  these  performance  measures  with  bincKuIar  gaze-errois 
were  alto  studied.  These  studies  were  performed  to  determine  how  accurate  binocular  fixation  must  be 
in  Older  to  insiue  rapid  and  accurate  tapping  in  nearby  3-D  space.  We  found  that  effective  performance 
was  possible  with  binocular  gaze-errors  about  the  size  of  the  human  fovea  (2**  -  3°). 
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Research  Objectives;  (Period  covered  12-15-90  to  03-31-94)  unlimited. 

This  grant  had  two  thrusts.  First,  it  tested  alternative  hypotheses  about  the  mechanism,  which 
controls  the  gaze-shifts  associated  with  arm  modems  when  a  seated  subject  manipulates  objects  within 
arms’s  reach  as  the  body  moves  naturally.  Two  quite  different  hypotheses  have  been  proposed, 
namely,  a)  on-line  feedback  and  b)  learned,  preplanned  patterns  of  coordinated  movements.  These 
patterns  might  be  learned  during  the  course  of  practice  with  a  novel  manipulative  task  or  brought  to 
the  task  by  means  of  a  previously  well-established,  well-calibrated  eye-head-torso-arm  nuq)  (or  maps). 
Or,  peiiiaps  more  plausibly,  a  combination  of  these  two  kinds  of  pre-existing  and  newly  acquired 
coordinated  patterns,  providing,  of  course,  that  on-line  feedback,  the  alternative  hypothesis,  is  not  the 
whole  story.  If  on-line  feedback  were  to  be  very  effective,  the  ability  to  modify  preplanned  patterns 
nq>idly  becomes  relatively  unimportant  to  the  organism  because  these  patterns  would  not  have  to  be 
modiried  when  the  body  is  free  to  move  and  the  environment  imposes  sudden  unexpected  demands 
(HI  the  visuomotor  and  vestibular  systems. 

The  second  thrust  of  the  grant  was  to  study  the  speed  and  accuracy  of  visually-guided  hand 
movements  and  the  correlation  of  these  performance  measures  with  bincmular  gaze-errors.  In  general 
terms,  how  well  must  you  fixate  a  3-D  pattern  of  targets  as  you  execute  a  visually-guided  arm 
movement  rapidly  and  accurately  while  moving  naturally  in  3-D  space  ?  A  number  of  specific 
questions  arise;  such  as;  How  does  binocular  visual  search  p  jceed  when  a  3-D  pattern  is  seen  for 
the  first  time;  How  does  this  pattern  change  when  the  pattern  of  targets  is  seen  again  on  the  second 
and  subsequent  trials  and  the  subject  knows  that  the  current,  randomly  chosen,  pattern  will  be  seen 
again;  Does  one  eye  fixate  nearby  3-D  targets  more  accurately  than  the  other  eye:  If  so,  does  this  kind 
of  eye  superiority  relate  to  eye  or  hand  superiority  in  other  tasks;  Does  tiie  subject  fixate  accurately 
before  reaching,  look  and  reach  together,  look  for  the  next  target  while  reaching  for  the  present  target 
and,  once  again,  how  do  tiiese  characteristics  change  as  the  task  is  repeated  a  number  of  times? 

The  answers  to  these  and  other  questions  were  not  known  because,  hitheitofore,  it  had  not 
been  possible  to  measure  binocular  gaze-errors  accurately  as  a  subject  manipulates  nearby  objects  in 
3-D  space  with  the  head  and  torso  free  to  move  naturally.  The  first  year  an<l  a  half  of  the  grant  were 
devoted  to  the  development  of  instrumentation  to  make  it  possible  to  begin  to  study  and  answer  these 
questions.  Instrumentation  developments  included  hardware  and  software  to  control  experiments  and 
collect  data  on  a  new  family  of  computers.  We  switched  from  DEC  PDF  1 1/23  micros  to  486/50 
MHz  PCs. 

These  changes  were  required  to  accommodate  the  enormous  amounts  of  data  that  would  be 
prcxluced  when  we  recorded  binocular  2-D  eye  and  3-D  head  movements  as  subjects  pefformed  rather 
natural  coordinated  head/eye/hand  tasks  with  their  heads  and  torsos  free  to  move  naturally.  We  also 
needed  to  develop  instrumentation  to  provide  the  subject  with  meaningful,  naturalistic  tasks  to 
perform  under  visual  guidance.  This  need  led  to  the  development  of  what  we  have  a  called  the 
"worktable"  —  an  environment  in  which  the  subject’s  visuomotor  performance  could  be  varied  and 
his  performance  monitored  quantitatively.  These  needs  were  meet  in  the  second  year. 

We  succeeded  in  making  the  first  measurements  of  this  kind  during  the  secemd  grant  year  (>  2(X) 
Mb  of  eye/head/hand  data).  Following  this  period  of  data  collection  it  became  necessary  to  develop 
software  to  analyze  the  data  collected.  This  task  was  intellectually,  as  well  as  technically  challenging 
because  it  is  not  simple  to  calculate  the  gaze  of  subjects  as  they  use  their  eyes  to  guide  movements 
of  the  arms  and  torsos  as  the  manipulate  objects  in  nearby  space.  This  led  to  a  rigorous  theoretical 
treatment  of  the  variables  underlying  the  directions  of  gaze  in  our  environment,  which  produced  a 
Technical  Report  (>  100  pages)  describing  the  theory  of  our  unique  instrument  and  the  ^propriate 
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way  to  treat  the  data  it  produced.  This  theoretical  treatment  made  it  possible  to  develop  the  analytic 
so^are  required  to  interpret  the  data  obtained.  This  kind  of  software  was  developed  (by  J.  Epelboim 
-  an  AASERT  student  supported  by  this  grant)  for  the  Open  Window  environment  of  a  SUN 
SpaicWoikstation2GS,  procured  on  this  grant  for  this  purpose.  This  kind  of  ty)plication  software  is 
never  "finished",  but  it  has  already  attained  a  fairly  high  degree  of  sophistication,  which  has  made 
it  possible  to  begin  to  answer  a  number  of  the  questions  raised  above. 

Answers  to  some  of  these  questions  were  disseminated  as  they  became  available  during  the 
sectmd  and  third  year  of  the  grantfviz..  The  aimual  ARVO  Meeting  in  Sarasota,  Florida,  May  1992; 
The  Dutch  Motor  Control  Meeting  in  Utrecht  in  December  1992;  The  Marco  Island  Neural  Control 
of  Movement  Meeting  in  April  1993;  The  armual  ARVO  Meeting  in  Sarasota  in  May  1993;  The 
ftitemational  Physiology  Congress  in  Glasgow  in  August  1993;  in  a  Wermer-Gren  Symposium  on 
Dyslexia  in  Stockholm  in  January  1994;  at  the  Neural  Control  of  Movement  Meeting  in  Mtui  in  April 
1994  and  at  the  annual  ARVO  Meeting  in  Sarasota  in  May  1994). 

So,  the  39.5  months  during  which  this  grant  was  operational  were,  as  anticipated,  very  busy, 
very  denuuiding  intellectually  and  technically  and  very  exciting.  They  were  also  very  stressful 
because  of  the  need  to  meet  a  firm  deadline  for  gathering  together  our  group  of  collaborators 
(Collewijn,  Erkelens,  Kowler  and  Pizlo)  in  College  Park  where  diey  would  work  with  Epelboim  and 
Steirunan  to  use  the  Maryland  RFM  to  collect  novel  data  with  this  unique  instroment  We  met  diis 
deadline,  ran  the  planned  experiments  and  succeeded  in  collecting  all  necessary  data  to  deal  with 
many  of  the  questions  mentioned  just  above.  This  meant  that  our  research  plan  proceeded  pretty  much 
on  schedule  and  our  objectives  were,  in  the  main,  met. 

PROGRESS:  Some  highlights  of  our  research  progress  are  as  follows: 

We  used  the  unique  apparatus  developed  on  this  grant  to  record  eye  and  head  movements 
of  subjects  as  they  tiq)ped  or  only  looked  at  sequences  of  2,  4  or  6  targets.  Each  sequence  was 
repeat^  10  times  to  allow  an  opportunity  for  the  3-D  pattern  to  be  learned.  A  stereotypical, 
coordinated  pattern  of  eye,  head  and  arm  movements  was  established  after  2-3  repetitions.  Subjects 
almost  always  looked  at  each  target  just  before  it  was  tapped.  Looldng-tmly  was  more  difficult  than 
uq)ping  in  that  it  took  more  time  and,  unlike  tapping,  usually  did  not  benefit  from  practice. 
Time/target  increased  with  the  number  of  targets,  a  finding  called  a  "sequence  length  effect"  in  the 
motor  literature.  Such  sequence  length  effects  suggest  that  movement  sequences  were  plarmed  as  a 
whole,  not  (me  at  a  time. 

The  facts  that  subjects  looked  at  targets  before  tq>ping  them  and  could  not  perform  the  task 
with  eyes  closed  show  die  importance  of  eye  movements,  allowing  a  foveal  view,  for  accurate 
tapping.  The  relative  difficulty  of  the  less  natural,  looking-(mly  task,  in  which  the  eyes  woriced 
without  a  meaningful  cognitive  or  motor  purpose,  suggests  that  motor  planning  is  hierarchical, 
occurring  first  at  the  level  of  the  task  and  then  ^  the  level  of  specific  motor  programs.  Looking, 
unlike  l(X)king  while  doing,  is  not  such  a  task,  ft  served  no  obvious  purpose  and  did  not  lend  itself 
readily  to  die  development  of  efficient  programs,  ft  seems  that  the  development  of  sutdi  programs 
requires  a  natural,  purposeful  task  of  the  kind  eye  movements  evolved  to  serve. 

We  also  found  in  an  experiment  designed  specifically  to  test  alternative  views  of  die  nature 
of  the  cmrective  mechanisms  available  to  compensate  for  perturbations  of  the  body  during  gaze-shifts 
(described  above)  that  prevailing  views  were  incorrect.  Specifically,  we  found  diat  VOR  was  not 
turned  off  during  gaze-shifts,  an  idea  introduced  by  Lauratis  and  Robins(m  in  1986,  when  they 
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proposed  that  some  other  mechanism  was  turned  on,  allowing  gaze-shifts  to  be  accurate  whnt  dte 
head  was  pushed.  Our  data  support  the  older  idea  introduced  by  Bizzi  et  aL  in  1971  that  VOR 
remains  active  during  gaze-shifts,  but  it  is  largely  ineffective,  ctmtrary  to  the  claim  of  Bizzi  et  al.. 
hi  short,  if  you  want  to  make  accurate  gaze-shifts  avoid  circumstances  in  which  you  must  depend  on 
compensatian  by  die  VOR  because  it  is  not  good  enough  to  compensate  for  appreciable  motion  of 
the  head. 

These,  and  other  related  findings  are  currently  being  written  up  for  publicatimi. 

Odier  Activities:  Long  periods  during  the  first  2  years  of  diis  grant  were  spent  waiting  for  hardware 
to  be  built,  tested  and  calibrated.  During  diis  time  collaborators  were  encouraged  to  use  our  facilities 
and  dieir  free-time  to  advance  science  in  ways  appropriate  to  dieir  particular  professional  skills.  This 
led  to  a  number  of  Imiadly-based  scientific  accomplishments.  Specifically: 

(a)  Epelboim,  collaborating  closely  with  Kowler  as  well  as  myself,  completed  a  series  of  experiments, 
showing  quite  clearly  that  the  low  velocity  oculomotor  control  subsystem  (slow  control)  is  not 
position  soisitive  and  seems,  dierefore  to  be  responsible  for  minimizing  drift  velocity  exclusively 
(ctMitrary  to  claims  that  have  been  made).  This  work  satisfied  ^wlboim’s  "masters  ot  researdi 
competency"  requirement  This  woric  was  published  in  Vision  Research,  following  a  report  at  ARVO. 

(b)  Pizlo  completed  his  Ph.D  thesis  on  s)uq>e  constancy  during  this  period,  describing  portions  of  diis 
work  both  at  ARVO  and  at  the  aimual  meeting  of  the  Mathematical  Psydiology  Society.  Pizlo  has 
continued  to  collaborate  (at  no  cost)  on  the  grant  projea  since  joining  the  Department  of 
Psychological  Sciences  at  Purdue  U.  in  August  and  we  look  forward  to  his  continuing  to  collaborate 
because  he  has  made  major  contributions  to  our  research  to  date.  A  portion  of  his  doaoral  thesis  is 
now  in  press  in  die  Computational  Section  of  Vision  Research  and  another  psqier  based  rni  this  work 
is  under  revision  for  publication  in  this  journal. 

(c)  L  in  collaboration  with  ^lelboim  and  Booth  (a  cognitive  graduate  student  and  Airey  (an 
undergraduate  honors  student),  worked  tm  reading  research  while  waiting  for  the  hardware  and 
software  development  described  above.  An  abstract  tm  reading  spaced  and  unspaced  text  was 
accepted  for  presentation  at  ARVO  in  May  1992  and  Julie  Epelboim  (the  first  author)  was  awarded 
a  travel  grant  to  permit  her  to  present  this  material  at  the  meeting.  We  have  a  very  long  ptqier  (hi 
reading  unspaced  text  in  press  in  Vision  Research. 


(d)  Marie  Edwards,  a  Fh.  D.  physicist  working  as  a  Research  Associate  on  the  grant  primarily  on  the 
tiiemetical  treatment  of  our  novel  instrumentatitm,  used  his  free-time  to  publish  a  number  of  things 
in  his  specialty. 
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Collewi^  H,  Steinman,  R,  M,  Erkelens,  C,  J,  Kowler,  E,  and  Van  der  Steen,  J,  Binocular  gaze  ctmtrol 
under  ftee-head  conditions.  In  VESTIBULAR  AND  BRAIN  STEM  CONTROL  OF  EYE, 
HEAD  AND  BODY  MOVEMENTS,  Edited  by  Shimazu,  H  and  Shinoda,  Y,  Karger,  Basel, 
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University  Press,  Lcmdon,  pp  419-426, 1992 
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of  vergence  and  veisitm.  In:  Ygge,  J.  and  Lennerstrand,  G.  (Eds.)  "Eye  Movements  in 
Reading".  Elsevier/Pergamcm  (Oxford),  (In  press),  1994. 
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